Enhancing Instruction of Written East Asian Languages with Sketch Recognition-Based “Intelligent Language Workbook” Interfaces

For American higher education students studying one of more of the major East Asian languages that are Chinese, Japanese, and Korean (CJK) as a second language, one of the major challenges that students face is the mastery of those languages’ various written scripts due to their vast contrasts from written English. Conventional pedagogical resources for written CJK frequently rely on languages instructors, whom provide in-person demonstrations of those languages’ written scripts and real-time assessment of students’ written attempts; papers workbooks, which offer guided instructional drills and supplementary knowledge on the written component; and practice sheets, which enable students to repetitiously practice writing areas of the written scripts for memorization purposes. Unfortunately, these techniques also present their own inherent disadvantages: language instructors are constrained by time to focus on the written component for typical classroom sizes, workbooks are static instructional materials lacking realtime intelligent feedback and assessment, and practice sheets result in monotonous practice to students and are vulnerable to students repeatedly practicing on potential writing mistakes if left unsupervised. In this paper, we describe our work behind “intelligent language workbook” interfaces which combine the benefits of stylus-driven tablet devices and state-of-the-art sketch recognition algorithms for developing intelligent computer-assisted instructional interfaces catered towards written CJK instruction. We evaluated our interfaces on their capabilities to provide instructoremulated feedback and assessment on the visual structure and writing technique of users’ input for several written scripts, and our findings demonstrate strong results for supporting the incorporation of educational applications supporting written CJK instruction.